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Cross Reference to Related Applications 
United States Letters Patent Serial No. 08/656,035 filed 
May 31, 1996 for a Method and Apparatus for Independent 
Operation of a Remote Data Facility which application is 
assigned to the Same Assignee as this application. 

United States Letters Patent Serial No. 08/842,953 filed 
April 25, 1997 by Yuval Ofek for a Method and Apparatus for 
Independent and Simultaneous Access to a Common Data Set, which 
application is assigned to the same Assignee as this 
application. 

Background of the Invention 
Field of the Invention 

This invention generally relates to redundant data storage 
devices in a data processing network and more particularly to a 
method and apparatus that enables data to cascade through 
multiple redundant data storage units. 

Description of Related Art 

The maintenance of data integrity by data redundancy has 
become a very important issue. Data redundancy has several 
forms or variations. At a single site, mirroring or RAID 
redundancy protects against disk or other storage failure. In 
another form described in United States Letters Patent Serial 
No. 08/656,035 filed May 31, 1996 and others, redundancy is 
achieved by duplicating a local system at a remote location. 
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The use of a remote location prevents data loss due to natural 
disasters and the like at one site. 

In accordance with the foregoing United States Letters 
Patent Serial 08/656,035, all data processing activity occurs 
at a "local" or "production" site that contains a host system 
for processing data stored in a data storage facility. A 
geographically remote or backup site includes a data storage 
facility as a "redundant" facility for maintaining a 
restoration or recovery data set. In this system each time the 
host at the production site writes data to the production site 
data storage facility, the production data storage facility 
automatically writes data to the remote storage facility. In 
many, if not most, of these applications, writing data to the 
remote site data storage facility requires a transfer across a 
high bandwidth communications link so the backup procedure does 
not affect operations at the production facility. T3 and ESCON 
lines are typically preferred as the communications links 
despite their expense. 

United States Letters Patent Serial No. 08/842,953 
discloses another concept involving a BCV device typically used 
at the production site. In this approach a data set, for 
example a "logical volume", on a production site data storage 
facility, i.e., a "production volume", operates in a normal 
fashion. Another logical volume at this same site is 
designated as a "BCV volume" . An ESTABLISH command connects 



E3 0-02 9CON2 (98-096CON2) 

the BCV volume to the production volume so that the BCV volume 
synchronizes with the production volume. A SPLIT command 
thereafter can separate the BCV volume from the production 
volume making the data stored on the BCV volume available for 
5 another application. 

This other application may alter the data stored on the 
BCV volume. Consequently whenever the BCV volume is 
reconnected with the production volume, it is necessary to 
transfer data to the BCV volume that represent changes in both 

10 the BCV volume and the production volume. An alternative is to 

reconnect the BCV volume to the production volume by issuing 
another ESTABLISH command. However, this command will replace 
all the data on the BCV volume so all the data from the 
production volume must be transferred to the BCV volume whether 

15 or not changes have occurred. 

In the past locating data storage facilities at a 
production site and a single remote site has been acceptable. 
In certain critical applications it is now also desirable, and 
in some situations mandatory, to store still another 

2 0 restoration copy at a third site that is remote from the first 

two sites. This requires some approach for copying the data 
from the production site to both remote sites, all 
transparently to the operations at the production site. Merely 
repeating the foregoing approach for copying data from a 

25 production site to a single remote site involves excessive 
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communications costs. A second high-speed communications link 
will be required between either the production site and the 
second remote site or between the first and second remote 
sites. What is needed is a way to establish a redundant data 
copy over a less costly communications link without any 
significant disparity in the data that exists at various 
remotely located sites such that all the data storage 
facilities are in synchronism or nearly in synchronism. 

Summary 

Therefore it is an object of this invention to provide a 
method and apparatus for establishing redundant data storage 
facilities that can communicate over less costly communications 
links . 

Another object of this invention is to provide a method 
and apparatus for achieving redundant disk storage at 
geographically remote sites on an economical basis. 

Still another object of this invention is to provide a 
method and apparatus for providing redundant data storage at 
two or more geographically remote sites using an economical 
communications link . 

Yet another object of this invention is to provide a 
method and apparatus for providing redundant data storage at 
two or more geographically remote sites using an economical 
low-bandwidth communications link. 
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Yet still another object of this invention is to provide a 
data processing network in which data processed at a production 
site is stored at two or more redundant storage sites with 
redundancy being achieved transparently to operations at the 
5 production site. 

In accordance with this invention, a data storage site 
remote from a data processing facility that alters data, such 
as a production facility, transfers data to another site remote 
from the data storage facility. The data storage facility at 

10 the remote site includes first and second data stores. The 

first data store receives data from the production site. A 
data change recorder identifies changes that the data 
processing facility makes in the first data store. A first 
operating control establishes a first operating mode during 

15 which the second data store receives data from the first data 

store according to the changes recorded in said change 
recording means. After this operation is complete, a second 
operating control can establish a second operating mode for 
copying data from the second data store to the other remote 

2 0 site according to the changes recorded in said data change 

recorder. 

In accordance with another aspect of this invention, data . 
is transferred between a production site and a remote site. 
The production site includes a host and production storage 
25 facility; the remote site, a remote storage facility including 
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a first data store and a second data store wherein the first 
data store receives data from the production facility on a 
track -by -track basis. The host can issue a plurality of 
cascade commands to enable the definition of a plurality of 
track status tables for identifying each track in the first 
data store .that the production facility changes. In addition, 
the host can establish first and second operating modes. In 
the first operating mode the second data store receives data 
from the first data store according to the changes recorded in 
the track status tables. During the second operating mode data 
from the second data store is copied to the remote storage 
facility according to the changes recorded in the track status 
tables at the time the second mode is established. 

Brief Description of the Drawings 
The appended claims particularly point out and distinctly 
claim the subject matter of this invention. The various 
objects, advantages and novel features of this invention will 
be more fully apparent from a reading of the following detailed 
description in conjunction with the accompanying drawings in 
which like reference numerals refer to like parts, and in 
which: 

FIG. 1 is a block diagram of a data processing network 
including a production facility and two geographically remote 
facilities; 
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FIG. 2 is a flow diagram that illustrates the transfer of 
commands from a production facility to a remote storage 
facility; 

FIG. 3 depicts the operation of the network in FIG. 1 in 
response to a DIFFERENTIAL SPLIT command; 

FIG. 4 depicts details of a procedure used in the 
operation depicted in FIG. 3; 

FIG. 5 depicts the operation of the network in FIG. 1 in 
response to a RE-ESTABLISH command; and 

FIG. 6 depicts another operation that occurs after 
processing a DIFFERENTIAL SPLIT command. 

Description of Illustrative Embodiments 
General Operation 

FIG. 1 depicts a data processing network 2 0 with equipment 
located at a local or production site 21, a first remote site 
22 and a second remote site 23. The first and second remote 
sites 2 2 and 23 typically will be geographically remote from 
the local production site 21 and from each other. However, as 
will become apparent, the equipment at the local production 
site 21 and first remote site 22 could be collocated. 

A first level of redundancy is achieved in the data 
processing network 2 0 through interactions between the local 
production site 21 and the first remote site 22. As known, a 
host 24, that includes one or more central processors and a 
main memory, operates on various programs. Periodically the 
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host 24 will effect a transfer through a host adapter 25 to a 
disk storage device. This disk storage device may have many 
physical disk drives organized into discrete sections for 
storing related information. These include files or other data 
blocks. In the context of the equipment manufactured by the 
assignee of this invention, a typical storage section is a 
logical volume comprising a number of contiguous disk tracks 
and transfers are made on a track -by -track basis. In FIG. 1 an 
Rl logical volume 26 is representative of the many logical 
volumes that normally are included in such a disk storage 
facility. As will also be apparent the host 24 retrieves any 
information it needs from such a production storage facility 
through the host adapter 25. Such systems are well known in 
the art. U. S. Letters Patent Serial No. 08/656,035 describes 
one such system. 

Although not shown, the local production site 21 may 
provide redundancy for the Rl logical volume 26. For example, 
the Rl volume may actually be mirrored or constituted by an 
array of logical volumes in any one of various RAID 
configurations thereby to prevent failure of a particular 
physical disk drive from interrupting operations by the host 
24. As will become apparent, this invention can be applied 
notwithstanding any redundancy scheme implemented at the local 
production site 21. 



8 



E3 0 - 0 2 9CON2 ( 98 - 0 96CON2 ) 



As is also known in the prior art and described in United 
States Letters Patent No. 5,544,347 to Yanai et al. and 
assigned to the same assignee of this invention, discloses 
another redundancy scheme that can be implemented by mirroring 
the data in the Rl logical volume 2 6 at the first remote site 
22. More specifically in this configuration the local 
production site 21 includes a remote adapter (RA) 27 that 
connects through a high speed communications link, such as an 
ESCON or T3 communications line, to a corresponding remote 
adapter 31 in the first remote site 22. In this configuration 
each time the host 24 writes data to the Rl logical volume 26, 
the remote adapter 27 responds by transferring that data 
through the high speed communications link to the remote 
adapter 31 in the first remote storage site 22 for transfer to 
an R2 logical volume 32. Thus each WRITE operation is 
reflected or conveyed to the R2 logical volume 32 that acts as 
a remote mirror with respect to the Rl logical volume 26. If a 
natural disaster strikes the local production site 21, the data 
is readily available in the R2 logical volume at the remote 
site. Moreover if a host 33 is located in the first remote 
site 22, it is possible for all operations to shift to the 
first remote site for continued operation without any 
undetected data losses. 

As previously stated, United States Letters Patent 
Application Serial No. 842,953 discloses a data processing 
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network that includes a BCV logical volume associated with a 
data storage facility. In accordance with this invention, a 
BCV/R1 logical volume 34 is included in the first remote site 
22. It can comprise any dedicated logical volume within the 
first remote site 22 preferably on a physical disk drive that 
is different from the physical disk drive that contains the R2 
logical volume 32. 

In accordance with this invention and as described in more 
detail later, the BCV/R1 logical volume 34 can be connected 
either to the R2 logical volume 32 or to a remote adapter 35. 
In a first operating mode, the BCV logical volume 34 
synchronizes with the R2 logical volume 32. In a second 
operating mode with the BCV/R1 logical volume 34 attached to 
the remote adapter 35, data will transfer over another 
communications link 36 to a remote adapter 37 in the second 
remote site 23 for transfer to an R2 logical volume 4 0 or other 
data receiver. In one embodiment of this invention, the second 
remote site 23 contains a host 41 and eliminates the need for 
the existence of the host 33 at the first remote site 22. Thus 
the second remote site 23 becomes the restoration site or 
secondary site for operating on the data if a natural disaster 
occurs at the local production site 21. 

The second remote site is shown as containing an optional 
BCV/R1 logical volume 42. As described more fully later, 
including this logical volume at the second remote site 23 
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could allow a replication of the function performed in 
accordance with this invention to a third remote site. 

In general terms, a remote site constructed in accordance 
with this invention will have the basic structure of the first 
remote site 22 that constitutes a data storage facility. It 
includes a first data store in the form of the R2 logical 
volume 32 for connection to the local production site 21 that 
can alter data. The BCV/R1 logical volume 34 constitutes a 
second data store. In response to a first command, the data 
store facility in the first remote site 22 operates in a first 
operating mode during which the R2 logical volume 32 receives 
data from the remote adapter 31 and thereby is responsive to 
changes made to the data in the Rl logical volume 26. 
Typically this is accomplished synchronously so a high-speed 
communications link 3 0 is necessary. In this operating mode 
the BCV/R1 logical volume 34 is considered to be operating in 
its BCV or first operating mode. 

In accordance with this invention, the BCV/R1 logical 
volume 34 can shift to a second, or Rl, operating mode once 
synchronism is achieved in the first operating mode. In the 
second operating mode the first remote site transfers data from 
the BCV/R1 logical volume 34 through the remote adapter 35, 
communications link 3 6 and remote adapter 37 to the R2 logical 
volume 40. The timing of shifts from the first to the second 
operating modes will be determined by a system operator. 



11 



E3 0-02 9CON2 (98-096CON2) 



However, shifts from the second operating mode to the first 
operating mode will generally be made after the data transfer 
to the R2 logical volume 40 is complete. Typically the 
interval between shifts to the second operating mode will be in 
terms of minutes, hours or even days depending upon the 
activity in the R2 logical volume 32. 

Still in accordance with this invention, each time the 
BCV/R1 logical volume 34 shifts to its second operating mode, 
only the data tracks that have been altered during the first 
operating mode are transferred to the R2 logical volume 40. If 
the local production site 21 makes repeated changes to a single 
track of the R2 logical volume 32 between successive shifts to 
the second operating mode, only one transfer will occur from 
the BCV/R1 logical volume 34 to the R2 logical volume 40. Such 
a reduction can lead to a reduction in the bandwidth 
requirements on the communications link 36. For example, it 
may be possible to reduce the communications link 36 to a level 
that will allow transfers over low bandwidth telephone lines or 
the Internet . 

Thus in accordance with this invention data changes made 
to the Rl logical volume 26 are replicated in the R2 logical 
volume 32 and then through the BCV/R1 logical volume 34 to the 
R2 logical volume 40. This process of sequentially 
transferring the data is characterized as cascading and is 
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controlled by a cascade command set processed by the host 24 at 
the local or production site 21. 

Cascade Command Processing 

More specifically, the system operator or an application 
program being processed at the local production site 21 can 
effect the cascading operation being performed at the first 
remote site 22. As known, the host 24 in FIG. 1 will produce 
various commands including commands in the cascade command set. 
These commands can be directed to any number of device or 
addresses including the host adapter 25. 

Now referring to FIGS. 1 and 2, when the host adapter 25 
receives a command, step 44 uses a conventional process to test 
and decode the command. The test analyzes syntax, context and 
other related parameters to determine whether a correct command 
has been received. Step 45 determines whether the command is 
one of commands in the cascade command set, that is, a cascade 
command. If it is not, the host adapter 25 uses a procedure 46 
to implement a corresponding function. When a valid cascading 
command is received, the host adapter 25 uses step 4 7 to 
transfer the command to the remote adapter 27. 

When the remote adapter 27 receives a command at step 50, 
step 51 merely transfers the command over the communications 
link 30 to the remote adapter 31 without any further analysis. 
The remote adapter 31 performs a similar process. That is, the 
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remote adapter 31 receives the command in step 52 and uses step 
53 to transfer the command to the device controller 43. 

After the device controller 34 receives the command from 
the remote adapter 31 in step 54, the device controller 34 
decodes the command in step 55 and processes that command in 
step 56. 

When the operation defined by the cascade command or other 
command has been completed, the device controller 43 generates 
an acknowledgement in step 60 for transfer to the remote 
adapter 31; steps 61, 62 and 63 represent the process for 
transferring that acknowledgement back across the 
communications link to the remote adapter 27 and the host 
adapter 25 for transfer to the host 24. 

Thus, this process allows a program running in the host 24 
to issue commands to the device controller 43 in the first 
remote site that in turn controls the operation of the BCV/R1 
logical volume 34. Steps 60 through 63 then provide the 
necessary feedback so that the host is aware of the completion 
of any such operation. As a result, there is no requirement 
for a host such as host 33 in FIG. 1, at the first remote site 
to implement this invention. 

First Remote Site 22 

Referring again to FIG. 1, the first remote site 22 
includes a number of status registers and tables and program 
modules for implementing this invention. A control module 7 0 
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includes modules for processing various commands including 
commands in the cascade command set including an EST module 71 
for processing an ESTABLISH command, a DS module 72 for 
processing a DIFFERENTIAL SPLIT command and an REEST module 73 
for processing a REESTABLISH command. The ESTABLISH and 
REESTABLISH commands also perform functions independently of a 
cascading operation. 

Each of the logical volumes includes a dedicated portion 
for maintaining information about the status of individual 
physical cylinders and tracks within those cylinders for a 
device. One such portion of the R2 logical volume 32 is 
constituted by a dedicated portion 74 that contains a status 
word for each track. Each status word contains information 
about the track and dedicates a number of bit positions that 
are available for various status functions. One of those bit 
positions will be assigned for use in the cascading process and 
that bit position will be replicated as a Protection Bits Table 
75 that is a 1 x n table where n equals the number of tracks in 
the R2 logical volume 32. 

The first remote site 22 also includes an R2 track status 
table 76 and a BCV/R1 track status table 77. Each of the track 
status tables 76 and 77 includes a plurality of rows, 
specifically four rows assigned to four mirror devices or 
volumes . In normal operations when the BCV/R1 logical volume 
34 attaches to the R2 logical volume 32, the bit positions in 
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the Ml row of the R2 track status table 76 correspond to tracks 
in the R2 logical volume 32; the bit positions in the M2 row, 
to tracks in the Rl logical volume 26/ and bit positions in the 
M3 row, to tracks in the BCV/R1 logical volume 34 acting in its 
BCV operating mode. Similarly the BCV/R1 track status table 77 
includes a plurality of mirror rows including bit positions in 
an Ml row for identifying track status for the BCV/R1 logical 
volume 34 and bit positions in the M2 row for the R2 logical 
volume 40. 

A PB bit register 80 identifies a selected bit position in 
the Protection Bit table 75 that will be used during a 
cascading operation. A session flag 81 indicates whether the 
system is operating in response to cascade commands. A copy 
program 82 copies data to and from different logical volumes in 
the first remote site 22 facility as described more fully 
later. 

Operation - First Operating Mode 
The first remote site 22 initially will be configured to 
operate in a first operating mode during which the copy program 
82 copies data from the first data store in the form of the R2 
logical volume 32 to the second data store in the form of the 
BCV/R1 logical volume 34 according to changes received at the 
remote adapter 31. This configuration remains in place until 
such time as the data on BCV/R1 logical volume 34 is identical 
to the data stored on the R2 logical volume 32; that is until 
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the two logical volumes are synchronized. Thereafter the first 
operating mode maintains that synchronism. 

Operation - Second Operating Mode 

At any time after such synchronism has been achieved, a 
second operating mode can be established when the host 24 
issues a DIFFERENTIAL SPLIT cascade command. Alternatively an 
optional host 33 at the first remote site could also generate 
such a command. Issuing the command begins a cascade session. 

In response, the device controller 43 will isolate the 
BCV/R1 logical volume 34 from the R2 logical volume 32 and 
enable the copy program 82 in FIG. 1 to transfer data from the 
BCV/R1 logical volume 34 through the remote adapter 35 to a 
data receiving device as represented by the second remote site 
23 in FIG. 1 including the R2 logical volume 40. At any time 
after those two volumes are synchronized, a REESTABLISH command 
will revert the system to the first operating mode by detaching 
the BCV/R1 logical volume 34 from the remote adapter 35 and 
reattaching it to the R2 logical volume 32 after which the copy 
program 82 will resynchronize the BCV/R1 logical volume 34 to 
the R2 logical volume 32. The session continues by issuing 
subsequent DIFFERENTIAL SPLIT and REESTABLISH commands until a 
CLOSE SESSION command is generated. 

FIG. 3 depicts operation in response to the receipt of a 
DIFFERENTIAL SPLIT command in. more detail. When the host 
adapter 25 recognizes that a differential command has been 
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received in step 90, it tests the command for various possible 
error conditions as known and as described with respect to step 
44 in FIG. 2. If any errors exist, the procedure aborts using 
a reporting procedure as known in the art. If no errors exist, 
control passes to step 92 that transfers the DIFFERENTIAL SPLIT 
command for transfer through the remote adapter 27, 
communications link 3 0 and the remote adapter 31 to enable the 
control 70 to respond. 

When. the DIFFERENTIAL SPLIT (DS) module 72 responds in 
step 93, it performs a conventional, locking operation to 
preclude any other applications from effecting the selected 
logical volume while the lock exists. Step 94 monitors the 
session flag 81 to determine if a cascading session is in 
progress. If this is the first differential split operation of 
a cascading operation, the session flag 81 indicates that the 
session is not in progress. Control passes to step 95 that 
establishes the protection bits table 75 based upon, data stored 
at a predetermined location. For example, step 95 may retrieve 
a protection bits data file from a dedicated portion 74 in the 
R2 logical volume 32. Alternatively, step 95 may retrieve 
corresponding information from a copy of such a file that is 
maintained in memory. In whatever form, each entry corresponds 
to a track, and each entry contains a plurality of bit 
positions that can be assigned for arbitrary purposes. Step 95 
selects an unused bit position from a table (not shown) 
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identifying any valid uses of different bit positions to 
establish one bit position for the session that is beginning. 
Processes for selecting such a bit position are well known in 
the art. Then the DS module 72 records the selected bit 
5 position in PB BIT register 80. 

Step 96 represents a procedure for performing a 
conventional split operation using basic steps from a 
conventional module. Details of this operation are depicted in 
FIG. 4. Specifically, step 101 terminates the mirroring 

10 operation of the BCV/R1 logical volume 34 as a mirror for the 

R2 logical volume 32. This action prevents the copy program 82 
from transferring data from the R2 logical volume 32 to the 
BCV/R1 logical volume 34. However, each transfer to the R2 
logical volume 32 will be recorded in the M3 row of the R2 

15 track status table 7 6 thereby to indicate a change in the R2 

logical volume 26 that is not reflected in the BCV/R1 logical 
volume 34 . 

Step 102 manages WRITE PENDING operations as described in 
the foregoing United States Letters Patent Application Serial 

20 No. 08/842,953. Next step 103 copies any identification tables 

from the R2 logical volume 32 to the BCV/R1 logical volume 34 
so that the transfer of data from the first remote site 22 to 
the second remote site will include all the identification 
information. Step 104 then assigns the BCV/R1 logical volume 

25 34 to copy data to the R2 logical volume 4 0 at the second 



19 



E 3 0 - 0 2 9CON2 (98-096 CON2 ) 



remote site 23. This enables the copy program 82 to transfer 
data to the appropriate destination namely the R2 logical 
volume 40. 

When the operation of FIG. 4 completes, step 105 in FIG. 3 
sets all the bits in the M2 bit position of the BCV/R1 track 
status table 77. Setting all the bits establishes . an operating 
environment in which the data in all the tracks will be 
transferred to the R2 logical volume 40 in FIG. 1. Step 105 
also sets the session flag 81 so it will be clear that during 
any subsequent DIFFERENTIAL SPLIT the alternate path described 
later is followed. 

Step 106 releases the lock. Step 107 initiates the copy 
program 82 that begins the transfer of data from the BCV/R1 
logical volume 34 to the R2 logical volume 4 0 according to the 
data in the BCV/R1 track status table 77, particularly the 
information in the M3 row. As each copy operation occurs, the 
corresponding M2 bit in the BCV/R1 track status table 7 7 is 
cleared. When all the bits have been cleared, step 110 posts a 
complete status and a corresponding message is transferred back 
to the host adapter 25 thereby to indicate to the corresponding 
application program running in the host 24 that the transfer 
has been completed. 

Operation - Return to First Operating Mode 

At any time after all the data has been transferred to the 
R2 logical volume 40, the system operator can issue a 
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REESTABLISH command that constitutes another cascade command. 
Step 112 in FIG. 5 represents receipt of that command in the 
host adapter 25. Step 113 represents the context and syntax 
testing of that command with the possibility of aborting the 
process if an error is detected. Otherwise the host adapter 25 
uses step 114 to pass to the device controller 43 command for 
control transfer through the remote adapter 27 and 
communications link 30 to the remote adapter 31. 

The device controller 43 uses step 115 to indicate the 
receipt of such a command. Step 116 adds the BCV/R1 logical 
volume 34 as a local BCV mirror with the next available device 
mirror designation in the R2 track status table 76. In step 
117 the device controller 43 sets the BCV/R1 logical volume 34 
acting as a storage facility for the DIFFERENTIAL SPLIT 
operation to NOT READY (NR) . This prevents the COPY program 
from copying any further data to the second remote site 23. 
Any WRITE PENDING operations are set to an INVALID state in 
step 118. As described in the foregoing application, setting 
any WRITE PENDING operations does not affect the data being 
stored. Step 12 0 then merges any bits in the R2 track status 
table 76 that have been in the mirrored position allocated to 
the BCV/R1 logical volume 34 (i.e., the M3 bits) with the bits 
in the M4 bit position for the BCV/R1 track status table 77. 
During other applications the M4 bit position normally records 
any changes that are made to the data of the BCV/R1 device. 



E3 0 - 02 9CON2 ( 98 - 0 96CON2 ) 

However, no such changes are made during a differential split 
command when the system operates in the second mode. 
Consequently step 12 0 merely obtains the bits in the M3 row of 
the R2 track status table 76 representing any changes made to 
the R2 logical volume 32 while the first remote site was 
operating in the DIFFERENTIAL SPLIT operating mode. Next step 
121 releases the lock and step 122 posts a complete status that 
transfers back to the host adapter 25 to the host 24 in step 
123. Now the copy program 82 becomes available for 
transferring data from the R2 logical volume 32 to the BCV/Rl 
logical volume 34 according to the contents of the M3 bit 
positions in the R2 track status register table 76. During 
each transfer the copy program 82 clears the corresponding bit 
in the M3 bit positions of the R2 track status register table 
76. However, the corresponding bit position in the protection 
bits table 75 remains unchanged. 

Operation - Write to R2 Logical Volume 4 0 
•During the interval in which the data storage facility at 
the first remote site 22 operates in the second operating mode 
copying data from the BCV/Rl logical volume 34 to the R2 
logical volume 40, the host 24 can continue to issue writing 
operations to the Rl logical volume 2 6 and the R2 logical 
volume 32. The remote adapter 31 receives the WRITE data at 
step 130 at FIG. 6. The information is written immediately to 
the R2 logical volume 32 in step 131 and the corresponding 
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entries in the R2 track status table 76 are updated in step 
132. In this case, the corresponding track bit position in the 
M3 row would be updated to indicate that WRITE operation had 
caused a transfer to the R2 logical volume 32, but had not 
caused a corresponding change in the BCV/R1 logical volume 34 . 
Next the system would look to the session flag 81 to determine 
if a cascade operation were underway. If it were, step 133 
would transfer control to step 134 thereby to write the 
corresponding status to the Protection Bits Table 75, so that 
the Projection Bits Table 75 and the M3 row in the track status 
table 76 would have identical information. Step 134 is 
bypassed if a cascading operation is not underway. Step 135 
then represents the process by which the control 7 0 completes 
the write operation. In the case of a normal operation the 
WRITE operation would complete by transferring the changed data 
to the BCV/R1 logical volume 34. If the BCV/R1 logical volume 
34 were no longer attached, the WRITE request would be made a 
WRITE PENDING request. 

Operation - Successive Differential Split Operations 
After some additional time interval, that again may be 
defined by a system operator and measured in minutes, hours or 
days, host 21 issues another DIFFERENTIAL SPLIT command. When 
the device controller 43 receives that command, its control 
again transfers to the procedure set forth in FIG. 3. In this 
case, however, the session flag 81 is set so step 94 diverts to 
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step 141 in a manner analogous to that described with respect 
to step 101 in FIG. 4, terminates the operation of the BCV/R1 
logical volume 34 as a mirror to the R2 logical volume 32. This 
disables any further action of the copy program 82 for the 
purpose of transferring data from the R2 logical volume 32 to 
the BCV logical volume 34. Step 142, like step 102, manages 
all WRITE PENDING operations. 

Next, the DS module 72 uses step 143 to copy the 
Protection Bits Table 75 to the M2 row in the BCV/R1 track 
status table 77. Then the DS module 72 clears the Protection 
Bits Table 75. After these operations, the M2 bit positions in 
the BCV/R1 track status table 77 indicate those tracks in the 
R2 logical volume 32 that were altered subsequent to a last 
DIFFERENTIAL SPLIT request being processed. Clearing the 
protection bits in the protection bits table 75 assures that 
upon unlocking the logical volumes 32 and 34 any WRITE PENDING 
operations are properly recorded by setting an appropriate bit 
in the Protection Bits Table 75 . 

Next control in the DS module 72 then shifts from step 143 
to step 106 to release the lock applied in ste;p 93 and to step 
107 whereupon the copy program 82 is enabled to transfer data 
from the BCV/R1 logical volume 34 to the R2 logical volume 40. 
However, only data in changed tracks will be transmitted. That 
is, it is the nature of this command that all DIFFERENTIAL 
SPLIT operations after a first operation in a session transmit 
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only incremental amounts of information corresponding to data 
in tracks that have been altered. Moreover the number of 
tracks of data actually transferred over the communications 
link 36 normally will be significantly less than the number of 
writing operations that occurs between successive DIFFERENTIAL 
SPLIT operations. For example, if during such an interval 
there have been ten writing operations to different areas on 
the same track of the R2 logical volume 32, the BCV/R1 device 
34 will only contain the latest information and so only one 
track writing operation will be required to update the R2 
logical volume 40 in the second remote site 23. When such 
repetitive writing operations occur to a single track, the 
bandwidth requirements for the communications link 36 can be 
dramatically reduced. It is generally found the requirements 
can be reduced sufficiently so at least the next lower level of 
communications bandwidth, such as a Tl line can be used. In 
many applications it may be possible to reduce the bandwidth 
requirement even further to a level at which the Internet can 
be used as a communications link 36. 

The sequence of DIFFERENTIAL SPLIT and REESTABLISH 
commands can continue on an iterative basis so long as it is 
desired to use the second remote site 23 as a repository for a 
redundant copy. Once there is no longer any need to maintain 
the redundant copy, the host can generate a predetermined CLOSE 
SESSION command. This cascade command releases the assignment 
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of the PB bit position in register 80 and clears the session 
flag 81 to complete the cascading operation. 

This invention has been described in terms of a preferred 
embodiment in which a local production site 21 is 
representative of apparatus for periodically altering data in a 
production data storage facility. In accordance with this 
invention a first remote site includes a first data store for 
mirroring the data at the production site. A second data store 
in the form of the BCV/R1 logical volume 34 has two operating 
modes. In one, the BCV/R1 logical volume 34 acts as a mirror 
for the R2 logical volume 32. In the other mode instituted by 
issuing a DIFFERENTIAL SPLIT or equivalent command, the BCV/R1 
logical volume 34 detaches from the R2 logical volume 32 and 
conveys data identified in the BCV/R1 track status table 77 to 
the R2 logical volume 4 0 or other means in the second remote 
site 23 that acts as a receiver for data from the BCV/R1 
logical volume 34 . 

A data change recording apparatus represented by the 
Protection Bits Table 75 records any changes to the R2 logical 
volume 32 while the contents of the BCV/R1 logical volume 34 
are being transferred to the R2 logical volume 4 0 during the 
second operating mode. When control shifts the BCV/R1 logical 
volume 34 back to the first operating mode, the data change 
recording identifies those tracks or other data blocks that 
must be transferred from the R2 logical volume 32 to the BCV/R1 



E3 0 - 02 9CON2 ( 98 - 0 96CON2 ) 



logical volume 34. This changed data recording further 
controls those tracks that are transferred from the BCV/R1 
logical volume 34 to the data receiving site such as the second 
remote site 23 when the system again shifts the operation of 
the BCV/R1 logical volume 34 to the second operating mode. 

There are many variations that can be made to the 
specifically disclosed embodiment shown in FIGS. 1 through 6. 
FIG. 1 for example, depicts a specific organization of tables 
and modules. It will be apparent that different configurations 
of the information could be substituted without detracting from 
the overall operation of the first. remote site 22. FIGS. 1 
through 6 further depict the operation in conjunction with a 
single logical volume. Generally a network will include 
multiple logical volumes, each of which could be handled in 
this same way by replicating the operation and apparatus shown 
in FIGS. 1 through 6. Moreover, the description of the various 
modules establishes specific sequences of events for purposes 
of explanation; other sequences might also be included with the 
elimination of some or addition of other operations such as are 
necessary for other ancillary operations. Such sequences can 
be under manual control so that each transfer between operating 
modes will be initiated by a system operator. Alternatively, 
all or portions of a cascading session might be under automatic 
control. Thus while this invention has been disclosed in terms 
of certain embodiments, it is the intent of the appended claims 
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to cover all such variations and modifications as come within 
the true spirit and scope of this invention. 

What is claimed as new and desired to be secured by- 
Letters Patent of the United States is: 
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